Community

TensorFlow For loop to set weights and biases

I recently saw the most recent tutorial on the Machine Learning Series ("Neural Network Model - Deep Learning with Neural Networks and TensorFlow") and I came up with a rather simple solution to the "NN For loop Problem" as I will call it for the sake of this post.

Basically we need to create a for loop that goes through each layer that can have a number of nodes and we need to set it's weights and biases.

By putting the layers into a "Layers" dictionary it will be much easier to manipulate them and set their values than to have to manually write the code to do so (e.x hidden_1_layer, hidden_2_layer etc...).

Anyways here is the code:

import tensorflow as tf

input_data = 2 # It was 784 (28*28) on the tutorial
Layers = {} # I will store all the layers to this dictionary
n_classes = 10 # Number of classes (Number of output weights)
topology = [input_data, 5, 5, 5, n_classes] # e.x [2,3,1] = 2 weights at InputLayer, 3 weights at Hidden_1_layer and 1 output weights on output Layer

# Go through the topology list and add a layer to the "Layers" dictionary with the amount of weights specified by the topology list.
for i in range(1,len(topology)):
	Layer = {'weights':tf.Variable(tf.random_normal([topology[i-1], topology[i]])),
					  'biases':tf.Variable(tf.random_normal(topology[i]))}

	Layers[i-1] = Layer

print(Layers[0]) # Hidden_1_layer
print(Layers[1]) # Hidden_2_layer
print(Layers[2]) # Hidden_3_layer
print(Layers[3]) # Output Layer

Now that gets rid of those parts of the code from the tutorial:

n_nodes_hl1 = 500
n_nodes_hl2 = 500
n_nodes_hl3 = 500

n_classes = 10

	hidden_1_layer = {'weights':tf.Variable(tf.random_normal([784, n_nodes_hl1])),
					  'biases':tf.Variable(tf.random_normal(n_nodes_hl1))}

	hidden_2_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2])),
					  'biases':tf.Variable(tf.random_normal(n_nodes_hl2))}

	hidden_3_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl2, n_nodes_hl3])),
					  'biases':tf.Variable(tf.random_normal(n_nodes_hl3))}

	output_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl3, n_classes])),
					  'biases':tf.Variable(tf.random_normal([n_classes]))}

In order to Sum the weights*hidden_layer_index and add the biases we can write a function that so that it is easier to write and then make a for loop to use that function to Sum the weights*hidden_layer_index and add the biases and then we can store the
values (or is it a tensor? I am bit confused as of what exactly is a tensor) to a dictionary so that we keep things neat.

note:
I am not an expert on python I mostly wright code in C++ but got into Python because I saw the Machine Learning series. I tested the code with a terminal and seems to work. I do not know if that is an ideal solution.

def neural_network_model

import tensorflow as tf

input_data = 2 # It was 784 (28*28) on at the tutorial
n_classes = 10 # Number of classes (Number of output weights)
topology = [input_data, 5, 5, 5, n_classes] # e.x [2,3,1] = 2 weights at InputLayer, 3 weights at Hidden_1_layer and 1 output weights on output Layer

def neural_network_model(data):
	Layers = {} # I will store all the layers to this dictionary
	LCompute = {} # Layers for Computation Graph
	# Go through the topology list and add a layer to the "Layers" dictionary with the amount of weights specified by the topology list.
	for i in range(1,len(topology)):
		Layer = {'weights':tf.Variable(tf.random_normal([topology[i-1], topology[i]])),
				 'biases':tf.Variable(tf.random_normal(topology[i]))}

		Layers[i-1] = Layer

		#Code for Computation Graph

		l = tf.add(tf.matmul(Layers[i-2] if i != 1 else data, Layers[i-1]['weights']) + Layers[i-1]['biases'])
		l = tf.nn.relu(l) if i != len(topology)-1 else l
		LCompute[i-1] = l

		return output

# Layers[0] = Hidden_1_layer
# Layers[1] = Hidden_2_layer
# Layers[2] = Hidden_3_layer
# Layers[3] = Output_layer

# LCompute[0] = l1
# LCompute[1] = l2
# LCompute[2] = l3
# LCompute[3] = output